A Deep Belief Network Based Machine Learning System for Risky Host Detection

To assure enterprise security, typically a SIEM (Security Information and Event Management) system is built to correlate security events from different preventive technologies and flag alerts. Analysts in a security operations center (SOC) investigate the alerts to decide whether the related hosts are malicious or not. However, the number of alerts is overwhelming which exceeds the SOC’s capacity to handle and the false positive rate is also really high. Consequently, there is a great need to reduce the false alarms as much as possible. Instead of detecting network intrusion from outside of the enterprise, this paper focuses on detecting compromised hosts within enterprise by an intelligent Deep learning system. Our system leverages alert information, various security logs and analysts’ investigation results in a real enterprise environment to identify hosts with high likelihood of being compromised. Text mining and graph-based method are used to generate targets and extract features. In order to validate the effectiveness of our model, other machine learning algorithms such as Multi-layer Neural Network, Deep Neural Network, Random Forest etc. are applied to the same enterprise data. The results indicate that the Deep Belief Network (DBN) performs much better than other algorithms and is 6 times more effective than the current rule-based system. What is more, due to its effectiveness, this compromised host detection system has been implemented in a real enterprise production environment, which includes data collection, label creation, feature engineering and host score generation. Keywords—machine learning system; deep belief network; risky host detection